This application is based upon and claims the benefit of priority from the prior Japanese Patent Applications Nos. 2000-307574, filed Oct. 6, 2000, and 2001-295098 filed Sep. 26, 2001, the entire contents of both of which are incorporated herein by reference.
The present invention relates to an image processing technology for using an image taken by an image pickup apparatus such as a video camera as an interface for inputting commands, etc.
A keyboard, mouse, controller, etc. are input devices often used for a computer, video game machine, etc. The operator inputs desired commands by operating these input devices to render a computer, etc. to execute processing according to the commands entered. Then, the operator sees images and listens to sound, etc. obtained as the processing results from a display device and speaker.
The operator enters commands by operating many buttons provided on the input device while watching a cursor shown on the display device.
Such operations greatly depend on operating experiences of the operator. For example, for a person who never touched the keyboard before, entering desired commands using the keyboard is quite troublesome and time-consuming, and prone to input errors due to mistyping from the keyboard. For this reason, there is a demand for a man-machine interface that will provide the operator with an easy way to operate.
On the other hand, with the progress of multimedia technologies, people in general households can now readily enjoy capturing images using a video camera into a computer, etc., editing and displaying the images on a display device. Such technologies are also used for personal authentication by analyzing images of a physical body such as a face, extracting characteristic parts thereof to identify individuals.
Conventionally, these images are used as information to be processed by a computer such as editing or analysis. However, images taken have not been used so far for a purpose such as entering commands to a computer, for example.
It is an object of the present invention to provide an image processing technology to use images taken by an image pickup apparatus, etc. as an input interface to enter commands, etc.
An image processor according to the present invention to solve the above-described problems comprises image capturing means for capturing a mirrored moving image partially including a moving target, image generating means for generating an object image expressing a predetermined object according to the movement of the target included in the mirrored moving image captured by the image capturing means and controlling means for combining the object image generated by this image generating means with the captured mirrored moving image and displaying the combined image on a predetermined display device.
The xe2x80x9ctargetxe2x80x9d refers to a remarked part of an object (person or matter, etc.) whose image is taken by an image pickup apparatus that supplies the image to an image processor.
Another image processor according to the present invention comprises image capturing means for capturing a mirrored moving image partially including a moving target, detecting means for detecting the target and the movement component thereof by detecting image features between the mirrored moving image at the actual time point and the immediately preceding mirrored moving image, image generating means for generating an object image expressing a predetermined object in such a way as to change according to the movement component of the target detected by the detecting means and controlling means for combining the object image generated by this image generating means with the captured mirrored moving image and displaying the combined image on a predetermined display device.
These image processors generate object images according to the movements of targets included in the mirrored moving image. That is, the movement, color, shape of the object image to be displayed on the display device and if there is a plurality of object images, which object image should be displayed, etc. are determined by the movement of the target. For example, if the target is the operator, the object is determined according to the movement of the operator. Thus, the mirrored moving image is available as a kind of input interface.
In these image processors, it is also possible to construct the image generating means to generate the object image in such a way as to follow the movement of the detected target.
It is also possible to comprise means for making preparations for executing required processing based on the generated object image according to the movement component of the target.
It is also possible to further comprise means for comparing a combined image obtained by combining the object image generated by the image generating means and the mirrored moving image at the actual time point, with a template image which is the image of part of the target included in the immediately preceding mirrored moving image, detecting the part of the combined image whose image feature is most resembling the template image and making preparations for executing required processing based on this object image when the image of the part of the detected combined image includes the object image.
By associating the object image with predetermined processing and further comprising means for executing the processing linked to the object image when the movement component of the target detected by the detecting means satisfies predetermined conditions, it is possible to execute processing using the movement of the target as an input.
It is also possible to construct the image processor so that the mirrored moving image includes a plurality of targets, construct the detecting means to detect the movement components of the plurality of targets and detect one target based on the respective movement components of the detected plurality of targets, construct the image generating means to change the object image according to the movement component of the one target detected by the detecting means.
The present invention further provides the following image processing method. This image processing method comprises the steps of capturing a mirrored moving image partially including a moving target in the image processor, generating an object image expressing a predetermined object using the image processor according to the movement of the target included in the captured mirrored moving image and combining the object image generated with the captured mirrored moving image and displaying the combined image on a predetermined display device.
The present invention also provides the computer program. This computer program renders a computer with a display device connected thereto to execute the steps of capturing a mirrored moving image partially including a moving target, generating an object image expressing a predetermined object according to the movement of the target included in the captured mirrored moving image and combining the object image generated with the captured mirrored moving image and displaying the combined image on the display device.
The present invention also provides the semiconductor device. This semiconductor device renders a computer with a display device connected thereto to form the functions of means for capturing a mirrored moving image partially including a moving target, means for generating an object image expressing a predetermined object according to the movement of the target included in the captured mirrored moving image and means for combining the object image generated with the captured mirrored moving image and displaying the combined image on the display device.